Certain types of product development fall into certain patterns of use over time. As those patterns are discovered and studied, simulation models are oftentimes developed which provide structure and guidance to developers. One particular example of such a patterned and studied scenario is software and systems development.
Software development is understood both to be well suited to modeling and to being facilitated by consultation of models as development proceeds. Because software is frequently designed and studied in the abstract, and then coded and tested, the development process lends itself to a discretized pattern that can more easily be represented in simulation than other more heuristic types of development. One example of simulation models that typify the ability to break down a software development process can be found in the generalized software process simulation models of U.S. patent application Ser. No. 10/838,494, which is herein incorporated by reference.
Academic work has been done on the development of simulation models for software development processes. Existing systems for software development simulation, however, while attempting to provide useful metrics of such types as time, quality, cost, and features of a particular development project, are somewhat myopically focused on prediction of end results. This is quite useful at the beginning of a project, when developers must determine their ability to meet deadlines, deliver product of a certain quality, or stay below cost. However the focus on end results does not always provide information of the particular granularity or specificity required for decision-making during a software development project. Thus, if a developer or manager suspects that a project may not be performing as successfully as desired, and even if he or she can measure how far off course the project is, existing systems do not provide information necessary for that person to find a suitable place in the remaining project stages to change course and how much of a correction to make.
Additionally, existing systems, many of which are based on statistical process control (“SPC”), do not always provide an easy measure of whether or not a project is performing satisfactorily or not. This is because, while SPC has proven useful in many statistically-appropriate development environments, software presents problems that make typical SPC methods not very helpful in measuring the success of an ongoing project. SPC methods rely on past development data in order to provide statistically-based control limits for a project; seeing that a given metric for a project has gone outside of those control limits indicates that the project is “out of control” and thus must be corrected. % However, software does not always provide the uniformity of data that is required to create useful control limits. Typically, in a typical manufacturing application, one operation is repeated many times, and data are stratified by operator, activity, and machine. This wealth of data provides control limits that a manufacturer can be confident in and use. However, software development frequently involves such variation of task, operator, and tools that it can be difficult or impossible to gain data and stratify it to create proper control limits. Thus, software control limits, created by typical SPC methods, can vary from ±15% of mean performance to limits that range from 0% to 400% of the mean. This variation in control limits is too large to be useful to developers. An SPC-controlled project could report that it is “consistent” with past performance (i.e. that it is within the control limits) and yet could be performing unacceptably from a manager's standpoint, if the limits happen to be exceedingly wide.
What is needed is a system that provides practical information to determine the success of ongoing projects while also providing data which can indicate, in the event that a project is out of control, where resources should be reallocated to put the project back into control.